Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added post_install in setup.py file #3883

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

christinestraub
Copy link
Collaborator

@christinestraub christinestraub commented Jan 22, 2025

  • Add auto-download for NLTK for Python Enviroment When user install python library without image, It will automatically download nltk data.

  • Added entry_points in setup in setup.py

@christinestraub christinestraub marked this pull request as ready for review January 23, 2025 15:13
setup.py Outdated Show resolved Hide resolved
@christinestraub christinestraub added this pull request to the merge queue Jan 24, 2025
- **Correctly patch pdfminer to avoid PDF repair**. The patch applied to pdfminer's parser caused it to occasionally split tokens in content streams, throwing `PDFSyntaxError`. Repairing these PDFs sometimes failed (since they were not actually invalid) resulting in unnecessary OCR fallback.


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra newline

@christinestraub christinestraub removed this pull request from the merge queue due to a manual request Jan 24, 2025
@@ -32,6 +27,30 @@ def check_for_nltk_package(package_name: str, package_category: str) -> bool:
return False


# We cache this because we do not want to attempt
# downloading the packages multiple times
@lru_cache()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the logic below we don't need cache. Further more (this is kind of pathological edge case but...) caching means we could ignore situations where between checks the nltk data is removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants